The Master-Slave Architecture for Evolutionary Computations Revisited

The recent availability of cheap Beowulf clusters has generated much interest for Parallel and Distributed Evolutionary Computations (PDEC) [1]. Another often neglected source of CPU power for PDEC are networks of PCs, in many case very powerful workstations, that run idle each day for long periods of time. To exploit efficiently both Beowulfs and networks of heterogeneous workstations we argue that the classic master-slave distribution model is superior to the currently more popular island-model [1]. The key features of a good PDEC capable of exploiting networks of heterogeneous workstations are transparency for the user, robustness, and adaptivity. Transparency is essential to both the user of the PDEC and the user of the workstation, as none want to deal with the other. One way to implement such a PDEC is as a screen-saver. Robustness is very important because evolutionary computations may execute over long periods of time during which different types of failures are expected: hard failures caused by network problems, system crashes or reboots, and soft failures that stem from the use of the workstation for other tasks (e.g. when the user deactivates the screen-saver). Finally, adaptivity refers to the capability of the PDEC to exploit new or compensate for lost computing resources (dynamical network configuration). The classical island-model is not designed to deal with these features, essentially because populations (demes) are tightly coupled with processing nodes. In contrast, the master-slave model has all required features. One issue that needs to be addressed, however, is its ability to scale with a large number of slave nodes, knowing that there is a communication bottleneck with the master node. In the rest of this short paper, we build a mathematical model of the masterslave and show that, given current Local Area Network (LAN) technologies, a quite large PDEC can be built before reaching this bottleneck. For real world applications, assuming that the time needed for fitness evaluation is the dominant time factor for evolutionary algorithms, the speedup of a master-slave system over that of a single processor can be modeled by NTf/Tp, where N is the population size, Tf is the time needed to evaluate the fitness of a single individual, and Tp is the time needed to evaluate all individuals using P processors. Possible distribution policies range from separating the population into P sets and sending each of them to a different slave, or sending the individuals one-by-one to available slaves until all are evaluated. Let S designate the average size of the sets that are sent to processing nodes, and C = N/PS the